Image processing apparatus and method, and computer readable storage medium

ABSTRACT

Input image data is divided into blocks so as to make adjacent blocks partially overlap image data, and the divided image data is filtered in the unit of block. An image processing apparatus and method are provided which can process image data of an arbitrary size at high speed independently from the capacity of a buffer memory, and a computer readable storage medium is provided which stores a program realizing such a method.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an image processing apparatusand method suitable for performing a filtering process and codingprocess of image data, and to a computer readable storage medium storingprocess for realizing such a method.

[0003] 2. Relata Background Art

[0004] Digital still cameras are widely used as an image pickup devicefor computers. An image compression method utilizing DCT (discretecosine transform) such as JPEG is used for digital still cameras. Whiledigital still cameras are prevailing, high speed continuousphotographing and power saving have been desired. In order to shortenthe time taken to pickup images and record image data, the processesfrom signal processing to image compression have been conductedheretofore by hardware.

[0005] With a conventional hardware configuration, a capacityproportional to the input image scan size in a horizontal direction isrequired for a buffer which is used for filtering accumulated chargeinformation read from image pickup elements such as CCD in horizontaland vertical directions and for a buffer which is used for raster/blockconversion of raster scan sequential image data into block scansequential image data. For example, assuming that input image data has1034 pixels in the horizontal direction and 770 pixels in the verticaldirection, a horizontal filter has 11 taps, a vertical filter has 3taps, a YUV sampling ratio of JPEG is 4:2:2, CCD image data has 10 bits,and each of YUV has 8 bits, the following data is determined:

[0006] Hd=1034, where Hd is the number of pixels of input image data inthe horizontal direction;

[0007] Hr=1034−INT(11/2)×2=1024, where Hr is the number of pixels ofimage data in the horizontal direction output after signal processing (afiltering process, a YC separation, an edge process, a gamma process,and the like);

[0008] a capacity of a buffer for filtering in the horizontal andvertical directions =2×Hd×10=20680 (bits); and

[0009] a capacity of a buffer for raster/block conversion=8×Hr ((thenumber of bits of Y) +(the number of bits of UV)) =8×1024×16=131072(bits).

[0010] In the system using a plurality of image sizes, the buffercapacity has been determined conventionally in accordance with a maximumhorizontal image size, respectively for a buffer for filtering in thehorizontal and vertical directions and for a buffer for raster/blockconversion. This is not economical.

[0011] Conventional techniques are associated with a problem that when aCCD sensor of non-square is used, the compressed image data hasdistortion.

[0012] There is also a problem that since image data after signalprocessing is JPEG compressed, it is difficult to resize the image data.

SUMMARY OF THE INVENTION

[0013] Under the above-described background of the invention, it is anobject of the present invention to provide an image processing apparatusand method capable of processing image data having an arbitrary size athigh speed independently from the capacity of a buffer memory, and acomputer readable storage medium storing processes realizing such amethod.

[0014] In order to achieve the above object of the invention, accordingto one aspect of the present invention, there is provided an imageprocessing apparatus/method wherein input image data is divided intoblocks adjacent blocks of which partially overlap the image data, andthe divided image data is subject to a filtering process in a block unitbasis.

[0015] According to an another aspect of the present invention, there isprovided readable storage medium which stores a program executing stepsof inputting image data and dividing the input image data into blocksadjacent blocks of which partially overlap the image data.

[0016] Other objects, features and advantages of the invention willbecome apparent from the following detailed description taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017]FIG. 1 is a block diagram showing the structure of an imageprocessing apparatus according to a first embodiment of the invention.

[0018]FIG. 2 is a diagram showing image data stored in RAM 4.

[0019]FIG. 3 is a diagram illustrating the operation of atwo-dimensional DMAC.

[0020]FIG. 4 is a diagram illustrating a method of dividing input data,the method being executed by a signal processing circuit.

[0021]FIG. 5 is a diagram showing an effective image area of eachdivided area.

[0022]FIG. 6 is a diagram illustrating an overlap of blocks in thehorizontal direction.

[0023]FIG. 7 is a diagram illustrating an overlap of blocks in thevertical direction.

[0024]FIG. 8 is a diagram illustrating output data of the signalprocessing circuit.

[0025]FIG. 9 is a block diagram illustrating a clock control of thesignal processing circuit, the clock control being executed by a processSSG circuit.

[0026]FIG. 10 is a block diagram showing the structure on an imageprocessing apparatus according to a second embodiment of the invention.

[0027]FIG. 11 is a diagram illustrating a phase relation of input andoutput data during enlargement and reduction.

[0028]FIG. 12 is a block diagram showing the structure of an imageprocessing apparatus according to a third embodiment of the invention.

[0029]FIG. 13 is a block diagram showing the structures of an enlargingand reducing circuit and phase buffers according to the thirdembodiment.

[0030]FIG. 14 is a block diagram showing the fundamental structures ofthe enlarging and reducing circuit and phase buffers.

[0031]FIG. 15 is a timing chart illustrating the operation of controlsignals output from a process SSG circuit.

[0032]FIG. 16 is a block diagram showing the structure of an imageprocessing apparatus according to a fourth embodiment of the invention.

[0033]FIG. 17 is a block diagram showing the structures of an enlargingand reducing circuit and phase buffers according to the fourthembodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034]FIG. 1 is a block diagram showing an image processing apparatusaccording to the first embodiment of the present invention. A ROM 14 isa storage medium for storing a program which is used by a CPU 5 toexecute a process to be described later. This storage medium may be asemiconductor memory, an optical disk, a magneto-optic disk, a magneticmedium or the like. These storage media may also be a non-volatilememory card, a DC-ROM, a floppy disk, a magnetic card or the like.

[0035] Referring to FIG. 1, an image focussed upon a CCD 1 by an opticalsystem not shown is converted into accumulated charge information whichis converted by an A/D converter 2 from analog signals into digitalsignals. The digital signal is transferred via a CPU bus 3 to a RAM 4.If frame read of CCD 1 is performed, the digital accumulated chargeinformation is stored in RAM 4 as shown in FIG. 2.

[0036] CPU 5 sets predetermined parameters to a signal processingcircuit 9, a process SSG circuit 7 and a JPEG compression circuit 11,and instructs the process SSG circuit 7 to process signals. Thepredetermined parameters include an image data location of a memory, animage size, color filter information of CCD 1, a gamma correction value,an image compression set value and the like.

[0037] The process SSG circuit 7 sets four values, including a DMAtransfer start address (adr), a horizontal transfer number (m), avertical transfer number (n), and a vertical offset value (offset) to atwo-dimensional DMAC (direct memory access controller) 6 to thereby readimage data from RAM 4.

[0038]FIG. 3 illustrates the read sequence of image data from RAM 4assuming that adr=0×030 (hereinafter, numerals added with 0× arehexadecimal numerals), m=0×8, n=0×5, and offset=0×010.

[0039] The two-dimensional DMAC 6 holds adr therein as a read startaddress, and resets an internal counter for holding the horizontal readnumber and another internal counter for holding the vertical readnumber, to “0”. Each time data is read, the counter for holding thehorizontal read number is incremented by “1”.

[0040] When the count of the counter for holding the horizontal readnumber coincides with m, the data read start address is incremented byoffset, the counter for holding the horizontal read number is reset to“0”, and the counter for holding the vertical read number is incrementedby “1”. Data read is terminated after the counter for holding thevertical read number coincides with n. In this manner, by sequentiallychanging the read address, data is read from a rectangular area of RAM 4as shown in FIG. 3.

[0041] The process SSG circuit 7 repetitively controls thetwo-dimensional DMAC 6 to divide image data in RAM into a plurality ofblocks and read blocks 0, 1, 2, . . . pg-1 in this order from RAM 4 asshown in FIG. 4.

[0042] Referring to FIG. 4, H and V represent the horizontal andvertical sizes of image data of one from transferred from CCD 1 to RAM4. The image data is divided into p×q blocks, p blocks in the horizontaldirection and 1 blocks in the vertical direction, with hatched portionsbeing overlapped.

[0043] Image data read from RAM 4 in the above manner is sent via thecpu bus 3 to a delay circuit 8 and signal processing circuit 9. Astorage capacity of the delay circuit 8 is:

[0044] 2 (lines)×Hb×(the number of bits expressing one pixel).

[0045] Each block shown in FIG. 4 is constituted of four areas as shownin FIG. 5.

[0046] Referring to FIG. 5, an area 4-A is an area wherein image dataread from RAM 4 is valid and image data read from the delay circuit 8 isinvalid. Therefore, in this area, an output from the signal processingcircuit 9 is also invalid. “Vdelay” is equal to the number of delayedlines. In this embodiment, the Vdelay is “2” assuming that the signalprocessing circuit 9 uses a filter with three vertical taps.

[0047] Areas 4-B and 4-D are areas wherein although image data read fromRAM 4 and delay circuit 8 is valid, an output of the signal processingcircuit 9 is invalid because horizontal image data of two lines exist inthe filters of the signal processing circuit 9. If it is assumed thatthe signal processing circuit 9 is formed by a circuit constituted of adelay circuit and a filter having the number of NTap of taps in thehorizontal direction, the length of the areas 4-B and 4-D in thehorizontal direction is NTap/2. For example, the horizontal length is“5” if NTap is “11” because of rounding off the numerals lower than thedecimal point. In this embodiment, NTap is “11” so that the horizontallength Hfil=“5”.

[0048] An area 4-C is an area wherein an outoput of the signalprocessing circuit 9 is valid. The horizontal length Ha of this area 4-Cis set to a multiple of a horizontal direction size of MCU (minimumcoded unit (MCU): minimum data unit of JPEG), and the vertical length Vais set equal to the vertical direction size of MCU. Va is therefore “8”for a thinning-out ratio of 4:2:2 of JPEG.

[0049] As described above, an output of the signal processing circuit 9becomes valid only in a partial area of each block. As shown in FIG. 4,the process SSG circuit 7 controls the two-dimensional DMAC 16 in such amanner that the area 4-A overlaps the area 4-C of the upper block andthe area 4-B overlaps the area 4-C of the left block.

[0050] The overlap in the horizontal direction is shown in FIG. 6, andthat in the vertical direction is shown in FIG. 7.

[0051] Referring to FIG. 6, blocks X and X+1 are overlapped in thehorizontal direction. In this case, Hoffset=Ha.

[0052] Referring to FIG. 7, blocks X and X+p are overlapped in thevertical direction. In this case, Voffset=Va.

[0053] As shown in FIG. 8, the signal processing circuit 9 processesimage data of one frame and supplies a raster/block converter 10 withoutput blocks 0, 1, . . . pq-1 in this order. However, image data outputfrom the signal processing circuit 9 is smaller than the whole inputimage data by Hfil at right and left end portions in the horizontaldirection and by Vdelay lines at the upper end portion in the verticaldirection.

[0054] The sixe of each output block is Ha in the horizontal directionand Va in the vertical direction. The raster/block converter 10sequentially and continuously converts these output blocks and outputssignals equivalent to those when image data not divided into blocks issupplied. A storage capacity of a line buffer in the raster/blockconverter 10 is therefore sufficient if such the capacity has Ha linesin the horizontal direction and Va lines in the vertical direction.

[0055] Next, a JPEG compression circuit 11 compresses by JPEG method,image date raster/block converted by the raster/block converter 10, andwrites the compressed image data in RAM 4 via the CPU bus 3. Uponreception of an instruction from CPU 5, the JPEG compressed data writtenin RAM 4 is recorded in a recording medium 13 via the CPU bus 3 and anI/F 12 in accordance with a file record format such as FAT.

[0056] In this embodiment, the buffer capacities required for the delaycircuit 8 and raster/block converter 10 are calculated in the followingmanner, assuming that input image data has 1034 pixels in the horizontaldirection and 770 pixels in the vertical direction, the horizontalfilter of the signal processing circuit 9 has 11 taps, the verticalfilter has three taps, the thinning-out ratio of JPEG is 4:2:2, CCDimage data has ten bits per pixel, YUV each has eight bits per pixel,and an image is divided into four 6 blocks in the horizontal directionand 96 blocks in the vertical direction:

[0057] Ha=(1034−Hfil×2)/4 (division)

[0058]  =(1034−5×2)/4

[0059]  =256

[0060] Va=8

[0061] Vb=Va+2

[0062]  =10 bits

[0063] Hb=Ha+Hfil×2

[0064]  =256+5×2

[0065]  =266

[0066] therefore, the buffer capacity of the delay circuit 8 is

[0067] 2×Hb×10 bit=5320 bits; and

[0068] the buffer capacity of the raster/block converter is

[0069] Va×Ha×((number of bits of Y)+(number of bits ofUV))=8×Ha×((number of bits of Y)+(number of bits of UV))=8×256×16 bits=32768 bits.

[0070] The embodiment can therefore reduce the buffer capacity more thanthat used by the conventional process described earlier.

[0071] While image data is read from the area 4-A of RAM 4 shown in FIG.5, the output of the signal processing circuit 9 is always invalid andthe signal output from the delay circuit 8 to the signal processingcircuit 9 is also invalid. Therefore, as shown in FIG. 9, the processSSG circuit 7 may control clock signals which drive the signalprocessing circuit 9, and during the period while image data in the area4-A is read from RAM 4, a supply of clock signals to the signalprocessing circuit 9 is stopped by a switch 15. Even if such a clockcontrol is performed for the signal processing circuit 9, valid outputsignals are not affected at all because the clock control is performedduring the period while invalid signals are output.

[0072] In this embodiment, although parameters and two-dimensional DMAC6 are used, the process SSG circuit 7 may control a usual DMAC to readimage data from RAM 4 in the manner similar to the embodiment.

[0073]FIG. 10 is a block diagram showing an image processing apparatusaccording to the second embodiment of the invention. In this embodiment,switches 19 and 20 and an enlarging and reducing circuit 17 are added tothe first embodiment apparatus. In FIG. 10, same elements as those shownin FIG. 1 are represented by using identical reference numerals.Accumulated charge information of CCD 1 is A/D converted and stored inRAM 4 as shown in FIG. 2.

[0074] A process SSG circuit 7 sets four values including a DMA transferstart address adr, a horizontal transfer number m, a vertical transfernumber n, and a vertical offset value offset, to a two-dimensional DMAC6 to thereby read image data from RAM 4.

[0075]FIG. 3 illustrates the read sequence of image data from RAM 4assuming that adr =0×030 (hereinafter, numerals added with 0x arehexadecimal numerals), m=0×8, n=0×5, and offset=0×010.

[0076] The two-dimensional DMAC 6 holds adr therein as a read startaddress, and resets an internal counter for holding the horizontal readnumber and another internal counter for holding the vertical readnumber, to “0”. Each time data is read, the counter for holding thehorizontal read number is incremented by “1” and the read address isalso incremented by “1”.

[0077] When the count of the counter for holding the horizontal readnumber coincides with m, the data read start address is incremented byoffset, the counter for holding the horizontal read number is reset to“0”, the counter for holding the vertical read number is incremented by“1”, and the read start address is set as the read address. Data read isterminated after the counter for holding the vertical read numbercoincides with n. In this manner, by sequentially changing the readaddress, data is read from a rectangular area of RAM 4 as shown in FIG.3.

[0078] The process SSG circuit 7 repetitively controls thetwo-dimensinal DMAC 6 to divide image data in RAM into a plurality ofblocks and read blocks 0, 1, 2, . . . pg-1 in this order from RAM 4 asshown in FIG. 4. In this manner, the image data read from RAM 4 issupplied via the CPU bus 3 to a delay circuit 8 and a signal processingcircuit 9.

[0079] Each of blocks 0 to pq-1 is constituted of four areas as shown inFIG. 5, and each block is featured as described earlier.

[0080] Next, the enlarging and reducing circuit 17 enlarges or reducesan output of the signal processing circuit 9 at an enlargement orreduction factor designated by CPU 5. The structure of this enlargingand reducing circuit 17 is disclosed, for example, in Japanese PatentApplication No. 5-227414. A method of calculating data to be output fromthe enlarging and reducing circuit 17 changes according to thesequential order of input image data.

[0081] For example, FIG. 11 shows a phase relation between input andoutput data when image data is reduced by 4/9 and when image data isenlarged by 9/4.

[0082] In the case of reduction by 4/9, the phase of input data changesin nine periods and the phase of output data changes in four periods as:

[0083] Dout1=Din1

[0084] Dout2=3/4Din3+1/4Din4

[0085] Dout3=2/4Din5+2/4Din6

[0086] Dout4=1/4Din7+3/4Din8

[0087] Dout5=Din10

[0088] Similarly, in the case of enlargement by 9/4, the phase of inputdata changes in four periods and the phase of output data changes innine periods as:

[0089] Dout1=Din1

[0090] Dout2=5/9Din1+4/9Din2

[0091] Dout3=1/9Din1+8/9Din2

[0092] Dout4=6/9Din2+3/9Din3

[0093] Dout5=2/9Din2+7/9Din3

[0094] Dout6=7/9Din3+2/9Din4

[0095] Dout7=3/9Din3+6/9Din4

[0096] Dout8=8/9Din4+1/9Din5

[0097] Dout9=4/9Din4+5/9Din5

[0098] Dout10=Din5

[0099] At an M/N enlargement factor, (N+1) pieces of input data arerequired in order to output M pieces of data from the enlargement andreducing circuit 17. Therefore, Ha and Va defining the size of the area4-C are determined in the following manner.

[0100] Enlargement only in Horizontal Direction

[0101] If the enlarging and reducing circuit 17 enlarges an image by M/Nin the horizontal direction to resize or square the image, Ha and Va aredetermined so that the image data of Ha×Va processed by and output fromthe enlarging and reducing circuit 17 has a horizontal size which is amultiple of the MCU horizontal direction size and a vertical size equalto the MCU vertical direction size. Therefore, Ha takes a value providedby multiplying a least common multiple of M and the MCU horizontaldirection size by N/M and adding “1” to the multiplication result and Vatakes the MCU vertical direction size.

[0102] When only the horizontal direction is enlarged, both the switches19 and 20 are turned to the contact “1” side, so that the output of theenlarging and reducing circuit 17 is supplied via the switches 19 and 20to a raster/block converter 10.

[0103] For example, assuming that effective image data after signalprocessing for CCD 1 has 960 (horizontal) ×768 (vertical) pixels, theimage is enlarged by 16/15 in the horizontal direction to form imagedata having 1024 (horizontal)×768 (vertical) pixels, and this image datais JPEG compressed through thinning-out of 4:2:2, Ha and Va aredetermined in the following manner:

[0104] MCU horizontal direction size=16 pixels, and MCU verticaldirection size=8 pixels;

[0105] therefore

[0106] Ha=(least common multiple of 16 and 16)/16×15×n+1=15×n+1 (n is anatural number), and

[0107] Va=8.

[0108] The maximum value of n is determined from the buffer capacitiesof the delay circuit 8 and raster/block converter 10. For example,assuming that accumulated charge information of one pixel of CCD isconstituted of 10 bits, the delay circuit 8 has a buffer capacitycapable of storing two lines of horizontal 640 pixels the raster/blockconverter 10 has a buffer capacity capable of storing eight lines ofhorizontal 640 pixels, and Hfil is “5”, then:

[0109] n1=(640−1−Hfil×2)/15=41 (numerals lower than decimal point arerounded off);

[0110] n2=640(15×16/15)=40 (numerals lower than decimal point arerounded off); and

[0111] (maximum value of n)=(smaller one of n1 and n2)=40.

[0112] Each time the signal processing circuit 9 processes one block,the enlarging and reducing circuit 17 outputs to the raster/blockconverter 10 image data having (16×n) (horizontal)×8 (vertical) pixels,i.e., image data having a horizontal size which is a multiple of the MCUhorizontal direction size and a vertical size which is equal to the MCUvertical direction size.

[0113] Reduction only in Horizontal Direction

[0114] If the enlarging and reducing circuit 17 reduces an image by M/Nin the horizontal direction to resize or square the image, Ha and Va aredetermined so that the image data of Ha×Va processed by and output fromthe enlarging and reducing circuit 17 has a horizontal size which is amultiple of the MCU horizontal direction size and a vertical size equalto the MCU vertical direction size. Therefore, Ha takes a value providedby multiplying N/M by a least common multiple of M and the MCUhorizontal direction size, and Va takes the MCU vertical direction size.

[0115] When only the horizontal direction is reduced, both the switches19 and 20 are turned to the contact “1” side, so that the output of theenlarging and reducing circuit 17 is supplied via the switches 19 and 20to a raster/block converter 10.

[0116] For example, assuming that effective image data after signalprocessing for CCD 1 has 1060 (horizontal) ×768 (vertical) pixels, theimage is reduced by 28/29 in the horizontal direction to form image datahaving 1024 (horizontal)×768 (vertical) pixels, and this image data isJPEG compressed through thinning-out of 4:2:2, Ha and Va are determinedin the following manner:

[0117] MCU horizontal direction size=16 pixels, and MCU verticaldirection size=8 pixels;

[0118] therefore

[0119] Ha=(least common multiple of 28 and 16)/28×29×n=116×n (n is anatural number), and

[0120] Va=8.

[0121] The maximum value of n is determined from the buffer capacitiesof the delay circuit 8 and raster/block converter 10. For example,assuming that accumulated charge information of one pixel of CCD isconstituted of 10 bits, the delay circuit 8 has a buffer capacitycapable of storing two lines of horizontal 640 pixels, the raster/blockconverter 10 has a buffer capacity capable of storing eight lines ofhorizontal 640 pixels, and Hfil is “5”, then:

[0122] n1=(640−2×Hfil)/116=5 (numerals lower than decimal point arerounded off);

[0123] n2=640/(116×28/29)=5 (numerals lower than decimal point arerounded off); and

[0124] (maximum value of n)=(smaller one of n1 and n2)=5.

[0125] Each time the signal processing circuit 9 processes one block,the enlarging and reducing circuit 17 outputs to the raster/blockconverter 10 image data having (7×16×n) (horizontal)×8 (vertical)pixels, i.e., image data having a horizontal size which is a multiple ofthe MCU horizontal direction size and a vertical size equal to the MCUvertical direction size.

[0126] Enlargement in Vertical Direction by Devisor of MCU VerticalDirection Size

[0127] If the enlarging and reducing circuit 17 enlarges an image by adivisor of the MCU vertical direction size in the vertical direction, Hais determined in the manner similar to that in “Enlargement only inHorizontal Direction” or “Reduction only in Vertical Direction”.

[0128] The block vertical size Va is determined as in the following:

[0129] if the MCU vertical direction size is “8”, then

[0130] Va=8 at an enlargement factor “1”

[0131] Va=4 at an enlargement factor “2”

[0132] Va=2 at an enlargement factor “4”

[0133] Va=1 at an enlargement factor “8”; whereas if the MCU verticaldirection size is “16”, then

[0134] Va=16 at an enlargement factor “1”

[0135] Va=8 at an enlargement factor “2”

[0136] Va=4 at an enlargement factor “4”

[0137] Va=2 at an enlargement factor “8”

[0138] Va=1 at an enlargement factor “16”.

[0139] In this case, both the switches 19 and 20 are turned to thecontact “1” side, so that the output of the enlarging and reducingcircuit 17 is supplied via the switches 19 and 20 to raster/blockconverter 10. Each time the signal processing circuit 9 processes oneblock, the enlarging and reducing circuit 17 outputs to the raster/blockconverter 10 image data having a horizontal size which is a multiple ofthe MCU horizontal direction size and a vertical size equal to the MCUvertical direction size.

[0140] Reduction in Vertical Direction by 1/n

[0141] If the enlarging and reducing circuit 17 reduces an image by 1/n,Ha is determined in the manner similar to that in “Enlargement only inHorizontal Direction” or “Reduction only in Vertical Direction”.

[0142] Va is determined as:

[0143] Va=(MCU vertical direction size)×n.

[0144] In this case, both the switches 19 and 20 are turned to thecontact “1” side, so that the output of the enlarging and reducingcircuit 17 is supplied via the switches 19 and 20 to a raster/blockconverter 10. Each time the signal processing circuit 9 processes oneblock, the enlarging and reducing circuit 17 outputs to the raster/blockconverter 10 image data having a horizontal size which is a multiple ofthe MCU horizontal direction size and a vertical size equal to the MCUvertical direction size.

[0145] Enlargement/Reduction in Vertical Direction other thanEnlargement by Divisor of MCU Vertical Direction Size and Reduction by1/n

[0146] If the enlarging and reducing circuit 17 enlarges in the verticaldirection, an image by Mv/Nv other than a divisor of the MCU verticaldirection size and 1/n and enlarges or reduces the image by Mh/Nh in thehorizontal direction, Ha is determined as:

[0147] Ha=(multiple of Nh)+1 for enlargement; and

[0148] Ha=(multiple of Nh) for reduction.

[0149] Va is determined as a multiple of Va. In this case, CPU 5operates to make both the switches 19 and 20 be turned to the contact“0” side, so that the output of the signal processing circuit 9 isenlarged or reduced by the enlarging and reducing circuit 17 andthereafter temporarily stored in RAM 4. The image data written in RAM 4is divided in the unit of MCU vertical direction size and supplied viathe switch 20 the raster/block converter 10.

[0150] By determining Ha and Va and controlling the switches 19 and 20in the manner described above, an image data having a horizontal sizewhich is an integer multiple of the MCU horizontal direction size and avertical size which is equal to the MCU vertical direction size, isoutput to the raster/block converter 10.

[0151] The process SSG circuit 7 controls the two-dimensional DMAC 6 insuch a manner that areas of each block overlap as shown in FIG. 6 in thehorizontal direction and in FIG. 7 in the vertical direction.

[0152] In FIG. 6, Hoffset is:

[0153] Hoffset=Ha (for reduction); and

[0154] Hoffset=Ha−1 (for enlargement).

[0155] For the reduction, the areas 4-C of the blocks X and X+1 becomecontinuous in the horizontal direction, and for the enlargement, theareas 4-C of the blocks X and X+1 overlaps by one pixel in thehorizontal direction.

[0156] In FIG. 7, Voffest is:

[0157] Voffset=Va.

[0158] The areas 4-C of the blocks X and X+p become continuous in thevertical direction.

[0159] With the above operations, even if the image is divided andprocessed in the unit of block by the enlarging and reducing circuit 17,distortion of the image to be caused by phase differences is not formed.

[0160] In the second embodiment, Ha is determined from an enlargement orreduction factor. This factor may be determined from the buffercapacities of the delay circuit 8 and raster/block converter 10.

[0161] For example, assuming that the buffer capacity of theraster/block converter 10 is 320 pixels in the horizontal direction, atarget enlargement factor is 11/10, and the MCU horizontal directionsize is 16, the horizontal direction size Ha of the area 4-C of a blockwhich makes the buffer memory of the raster/block converter 10 full, is:

[0162] Ha=320×10/11=291 (numerals lower than decimal point are roundedoff).

[0163] In this case, the enlargement factor of the enlarging andreducing circuit 17 is set to 320/291 so that the number of horizontalpixels in an output of the enlarging and reducing circuit 17 is 320which satisfies the condition of a multiple of the MCU horizontaldirection size. Although this factor is different from the targetenlargement factor, this precision does not pose any practical problem.

[0164]FIG. 12 is a block diagram showing an image processing apparatusaccording to the third embodiment of the present invention. In thisembodiment, a phase buffer 21 is provided which is connected to aprocess SSG circuit 7 and an enlarging and reducing circuit 17. In FIG.12, the same elements as those shown in FIG. 10 are shown by usingidentical reference numerals.

[0165] Image information of CCD 1 stored in RAM 4 as shown in FIG. 2 isread by the process SSG circuit 7 and two-dimensional DMAC 6 in theorder illustrated in FIG. 3. In this case, in the two-dimensional DMAC6, the operations of an internal counter for holding the horizontal readnumber and another internal counter for holding the vertical readnumber, are performed in the manner similar to the second embodimentillustrated in FIG. 10. By changing the read address of RAM 4, imagedata is read from a rectangular area of RAM 4 as shown in FIG. 3.

[0166] As shown in FIG. 4, image data divided into a plurality of blocksis read from RAM 4 in the order of blocks 0, 1, 2, . . . , pg-1. Theimage data read from RAM 4 is supplied via a CPU bus 3 to a delaycircuit 8 and a signal processing circuit 9. Each of blocks 0 to pq-1 isconstituted of four areas as shown in FIG. 5.

[0167] Similar to the embodiment shown in FIG. 10, the enlarging andreducing circuit 17 calculates output data by a different methoddepending upon the sequential order of input image data.

[0168] For example, the phase relations between input and output data ata reduction factor of 1/9 and at an enlargement factor of 9/4 are set inthe manner as illustrated in FIG. 11.

[0169] As described earlier, in order for the enlarging and reducingcircuit 17 to output M pieces of data at an enlargement factor of M/N,N+1 pieces of input data are necessary. Therefore, depending upon thewidth of Ha, the phase in the enlarging and reducing circuit 17 maybecome discontinuous when the block changes, and image distortion isgenerated.

[0170] In order to avoid this, in this embodiment, the phase in theenlarging and reducing circuit 17 at the boundary of adjacent blocks andimage data are stored in the phase buffer. When the block changes, thephase at the boundary of geometrically adjacent blocks and image dataare read to perform an interpolation operation for enlargement andreduction.

[0171] The operation of storing and reading the phase and image data atthe block boundary will be described.

[0172]FIG. 13 shows the structures of the enlarging and reducing circuit17 and phase buffer 21. The enlarging and reducing circuit 17 isconstituted of a horizontal enlarging and reducing circuit 171 and avertical enlarging and reducing circuit 172, and the phase buffer 21 isconstituted of a horizontal phase buffer 211 and a vertical phase buffer212.

[0173]FIG. 14 shows the structures of an enlarging and reducing circuit66 constituting each enlarging and reducing circuit 171, 172 and a phasebuffer 67 constituting each phase buffer 211, 212. In FIG. 14, SIG-IN,STO, LOAD, and SEL are control signals supplied from the process SSGcircuit 7. The enlarging and reducing circuit 66 reads image data viaSIG-IN and an interpolation operation is performed by an interpolationoperation circuit 61. A phase for such the interpolation operation isobtained from the phase counter 62.

[0174] When STO becomes active (in the following description, thecontrol signal takes an active high level), image data, image data attwo points for linear interpolation, stored in the interpolationoperation circuit 61 is output via a DAT terminal of the interpolationoperation circuit and WEN (write enable) is made active. At this time,image data is written in a buffer memory 65 selected by a switch 64 inaccordance with the value of SEL. Also at this time, a count indicatinga current phase and output via a CNT terminal of the phase counter 62 isselected by the switch and written in the buffer memory 65.

[0175] When LOAD becomes active, the phase and image data stored in thebuffer memory 65 selected by a switch 63 in accordance with the value ofSEL are output to the phase counter 62 and interpolation operationcircuit 61. When LOAD becomes active, the interpolation operationcircuit 61 supplied the image data to the interpolation operationcircuit 61 via a LOAD-DAT terminal. When LOAD becomes active, the phasecounter 62 reads the phase via a LOAD-CNT terminal.

[0176] The control to be executed by the process SSG circuit 7 relativeto the horizontal enlarging and reducing circuit 171, horizontal phasebuffer 211, vertical enlarging and reducing circuit 172, and verticalphase buffer 212 operating in the above manner will be described withreference to FIG. 15.

[0177] In FIG. 15, HSEL, HSTO, and HLOAD are SEL, STO, and LOAD of thehorizontal enlarging and reducing circuit 171 and horizontal phasebuffer 211. VSEL, VSTO, and VLOAD are SEL, STO, and LOAD of the verticalenlarging and reducing circuit 172 and vertical phase buffer 212.

[0178] HLOAD is made active before the horizontal enlarging and reducingcircuit 171 processes image data at the top of a line of each block, andHSEL is set to the value same as the line number in the block of currentimage data. Therefore, the horizontal enlarging and reducing circuit 171reads from the horizontal phase buffer 211 the image data and phase atthe end of the same line in the preceding block.

[0179] HSTO is made active after the horizontal enlarging and reducingcircuit 171 processes image data at the end of a line of each block, andHSEL is set to the value same as the line number in the block of currentimage data. Therefore, the image data and phase at the end of thecurrent line is read out from the horizontal enlarging and reducingcircuit 171 and then written into the horizontal phase buffer 211.

[0180] VLOAD is made active while the enlarging and reducing circuit 172processes image data of the first line of each block, and VSEL is set tothe value indicating the horizontal pixel position of the image data tobe processed in the block. Therefore, the vertical enlarging andreducing circuit 172 reads from the vertical phase buffer 212 the imagedata at the same horizontal pixel position one line before.

[0181] VSTO is made active while the enlarging and reducing circuit 172processes image data of the last line of each block, and VSEL is set tothe value indicating the horizontal pixel position of the image data tobe processed in the block. Therefore, the image data and phase at thecurrent line is read out from the vertical enlarging and reducingcircuit 172 and then written into the vertical phase buffer 212.

[0182] The horizontal phase buffer 211 is reset when theenlargement/reduction is completed for the block p1, 2p-1, . . . , pq-1shown in FIG. 4, i.e., the last horizontal block. The vertical phasebuffer 212 is reset when the enlargement/reduction is completed for theblock pq-1 shown in FIG. 4, i.e., the last block of one frame.

[0183] As the process SSG circuit 7, enlarging and reducing circuit 17and phase buffer 21 operate in the manner described above,enlargement/reduction without distortion can be performed throughpipelining processings by hardware even if image data is divided.

[0184] An output of the enlarging and reducing circuit 17 is input to araster/block converter 10. The raster/block converter 10 performs araster/block conversion of the image data having a horizontal size whichis a multiple of the MCU horizontal direction size and a vertical sizeequal to the MCU vertical direction size. Ha and Va of the area 4-C aretherefore determined so that the image data enlarged/reduced by theenlarging and reducing circuit 17 has a horizontal size which is amultiple of the MCU horizontal direction size and a vertical size equalto the MCU vertical direction size.

[0185] The buffer capacity of the phase buffer 21 necessary for storingthe phase and image data is determined as in the following. For example,assuming that the image enlarged/reduced has 1024 horizontal pixels, onepixel is constituted of 16 bits, the phase is expressed by 8 bits, theMCU vertical direction size is 8, and the reduction is performed at areduction factor of 4/5 in the vertical direction, the buffer capacitiesare:

[0186] (16+8)×8×5/4=240 bits for the horizontal phase buffer 211; and

[0187] (16+8)×1024=24576 bits for the vertical phase buffer 212.

[0188]FIG. 16 is a block diagram showing an image processing apparatusaccording to the fourth embodiment of the invention. A different pointfrom the third embodiment is a provision of switches 19 and 20.

[0189] In this embodiment, as shown in FIG. 17, an enlarging andreducing circuit 17 and a phase buffer 21 are constituted of ahorizontal enlarging and reducing circuit 91, a vertical enlarging andreducing circuit 93, and a horizontal phase buffer 92.

[0190] If CCD 1 has non-square pixels, the enlarging and reducingcircuit 17 squares each pixel by enlarging or reducing only in thehorizontal direction. In this case, the block division of image data isperformed in the manner similar to the third embodiment, and CPU 5 turnsthe switches 19 and 20 to the contact “1” side.

[0191] The enlarging and reducing circuit 17 can therefore output animage having a horizontal size which is a multiple of the MCU horizontaldirection size and a vertical size equal to the MCU vertical directionsize. An output of the enlarging and reducing circuit 17 is input viathe switches 19 and 20 to a raster/block converter 10 to be raster/blockconverted, and thereafter JPEG compressed by a JPEG compression circuit11. The JPEG compressed data is written in RAM 4.

[0192] If the image data is not only squared but also resized, CPU 5turns the switches 19 and 20 to the contact “0” side. Of Ha and Va ofthe area 4-C of each block, Ha is set to an arbitrary value which doesnot exceed the allowable range determined by the buffer capacity of thedelay circuit 8, and Va is set as Va=Nv×n (n is a natural number) at anenlargement factor of Mv/Nv at the enlarging and reducing circuit 17.

[0193] As described with the third embodiment, of the enlarging andreducing circuit 17, the horizontal enlarging and reducing circuit 91realizes a continuity of the phase thereof by reading the precedingimage data and phase from the phase buffer 21, whereas the verticalenlarging and reducing circuit 93 realizes a continuity of the phasethereof by setting Va to a common multiple of Nv.

[0194] An output of the enlarging and reducing circuit 17 is written viathe switch 19 and CPU bus 3 into RAM 4. The image data enlarged/reducedand stored in RAM 4 is supplied via the CPU bus 3 and switch 20 to theraster/block converter 10 to be raster/block converted, and thereafterJPEG compressed by the JPEG compression circuit 11. The JPEG compresseddata is written in RAM 4.

[0195] As described above, according to the embodiments, image data isprocessed after it is divided into blocks. Therefore, the buffercapacity necessary for the succeeding processes such as processes by adelay circuit and a raster/block converter can be reduced.

[0196] Further, the horizontal pixel size allowing image signalprocessing does not depend upon the capacities of buffer memories to beused by the delay circuit and raster/block converter. Therefore, it ispossible to process image data having an arbitrary horizontal pixelsize. It is therefore possible to reduce the memory capacity and cost.

[0197] Image data having an arbitrary size can be processed andcompressed without using a memory such as RAM so that high speed signalprocessing is possible.

[0198] According to the embodiments, image data read from RAM or thelike and compressed through JPEG or the like can be processed throughhardware pipelining, so that high speed signal processing is possible.

[0199] Since the number of accesses to RAM can be reduced, a powerrequired for signal processing can be saved.

[0200] Image data having an arbitrary size can be enlarged or reducedindependently from the capacities of the buffer memories. It istherefore possible to reduce the memory capacity and cost.

[0201] Image squaring including signal processing and compressing can beperformed through hardware pipelining without using a memory such asRAM.

[0202] Still further, clocks used for signal processing can be stoppedduring a specific period so that a power required for signal processingcan be saved.

[0203] In the above embodiments, a JPEG scheme is used as a compressioncoding method. The invention is not limited only thereto, but isapplicable to all coding systems for executing coding on a predeterminedblock unit basis (e.g., a MPEG scheme).

[0204] In other words, the foregoing description of embodiments has beengiven for illustrative purposes only and not to be construed as imposingany limitation in every respect.

[0205] The scope of the invention is, therefore, to be determined solelyby the following claims and not limited by the text of thespecifications and alterations made with a scope equivalent to the scopeof the claims fall within the true spirit and scope of the invention.

What is claimed is:
 1. An image processing apparatus comprising: inputmeans for inputting image data; dividing means for dividing the inputimage data into blocks, said dividing means dividing the input imagedata so that adjacent blocks partially overlap the input image data; andprocessing means for filtering the image data divided by said dividingmeans in a block unit basis.
 2. An image processing apparatus accordingto claim 1, further comprising coding means for compression coding anoutput of said processing means.
 3. An image processing apparatusaccording to claim 2, wherein a horizontal size of the block isdetermined from a horizontal size of one frame of the image data and thenumber of taps of a horizontal filter to be used by said processingmeans, and a horizontal direction length of an overlapped portion of theblock is determined from the number of taps of the horizontal filter. 4.An image processing apparatus according to claim 3, wherein thehorizontal size of the block is determined from a horizontal size of aminimum processing unit of said coding means.
 5. An image processingapparatus according to claim 3, wherein a vertical size of the block isdetermined from the number of taps of a vertical filter to be used bysaid processing means and an operation mode of said coding means, and avertical direction length of an overlapped portion of the block isdetermined from the number of taps of the vertical filter.
 6. An imageprocessing apparatus according to claim 1, further comprising:converting means for enlarging or reducing an output of said processingmeans; and coding means for compression coding an output of saidconverting means.
 7. An image processing apparatus according to claim 6,wherein a horizontal size of the block is determined from a horizontalsize of one frame of the image data, the number of taps of a horizontalfilter to be used by said processing means and an enlargement factor, ina horizontal direction of said converting means, and a horizontaldirection length of an overlapped portion of the block is determinedfrom the number of taps of the horizontal filter and the enlargementfactor in the horizontal direction of said converting means.
 8. An imageprocessing apparatus according to claim 7, wherein a vertical size ofthe block is determined from the number of taps of a vertical filter tobe used by said processing means and an operation mode of said codingmeans, and a vertical direction length of an overlapped portion of theblock is determined from the number of taps of the vertical filter. 9.An image processing apparatus according to claim 1, further comprising:delay means for delaying the image data to be supplied to saidprocessing means; and control means for stopping an operation clock tobe supplied to said processing means, during a period while anoverlapped portion in a vertical direction is written in said delaymeans, sequentially from a first block.
 10. An image processingapparatus according to claim 2, wherein said coding means performs acoding process in conformity with a JPEG scheme.
 11. An image processingapparatus according to claim 6, wherein said coding means performs acoding process in conformity with a JPEG scheme.
 12. An image processingapparatus according to claim 1, wherein said dividing means includesmemory means for storing the image data, and the image data is dividedinto blocks by reading the image data from the memory means throughtwo-dimensional DMA.
 13. An image processing apparatus according toclaim 1, wherein said input means including image pickup means forpicking up an object image and outputting the image data.
 14. An imageprocessing apparatus according to claim 8, wherein said dividing meansincludes memory means for storing the image data, and the image data isread from the memory means in the block unit basis.
 15. An imageprocessing apparatus according to claim 14, wherein an output of saidconverting means is temporarily stored in the memory means and thensupplied to said coding means, in a case other than enlargement at anenlargement factor equal to a divisor of a vertical size of a minimumprocessing unit in a vertical direction of said coding means orreduction at a reduction factor of one divided by a positive integer.16. An image processing method comprising the steps of: inputting imagedata; dividing the input image data into blocks, said dividing stepdividing the input image data so that adjacent blocks partially overlapthe image data; and filtering the divided image data in a block unitbasis.
 17. A computer readable storage medium storing an imageprocessing program, the program comprising: an input process ofinputting image data; a dividing process of dividing the input imagedata into blocks, said dividing process dividing the input image data sothat adjacent blocks partially overlap the image data; and a process offiltering the divided image data in a block unit basis.